Search

Large language models (LLMs) have significantly advanced the field of artificial intelligence. Yet, evaluating them comprehensively remains challenging. We argue that this is partly due to the predominant focus on performance metrics in most benchmarks. This paper introduces CogBench, a benchmark that includes ten behavioral metrics derived from seven cognitive psychology experiments. This novel approach offers a toolkit for phenotyping LLMs' behavior. We apply CogBench to 35 LLMs, yielding a rich and diverse dataset. We analyze this data using statistical multilevel modeling techniques, accounting for the nested dependencies among fine-tuned versions of specific LLMs. Our study highlights the crucial role of model size and reinforcement learning from human feedback (RLHF) in improving performance and aligning with human behavior. Interestingly, we find that open-source models are less risk-prone than proprietary models and that fine-tuning on code does not necessarily enhance LLMs' behavior. Finally, we explore the effects of prompt-engineering techniques. We discover that chain-of-thought prompting improves probabilistic reasoning, while take-a-step-back prompting fosters model-based behaviors.

We study the in-context learning dynamics of large language models (LLMs) using three instrumental learning tasks adapted from cognitive psychology. We find that LLMs update their beliefs in an asymmetric manner and learn more from better-than-expected outcomes than from worse-than-expected ones. Furthermore, we show that this effect reverses when learning about counterfactual feedback and disappears when no agency is implied. We corroborate these findings by investigating idealized in-context learning agents derived through meta-reinforcement learning, where we observe similar patterns. Taken together, our results contribute to our understanding of how in-context learning works by highlighting that the framing of a problem significantly influences how learning occurs, a phenomenon also observed in human cognition.

Ecological rationality refers to the notion that humans are rational agents adapted to their environment. However, testing this theory remains challenging due to two reasons: the difficulty in defining what tasks are ecologically valid and building rational models for these tasks. In this work, we demonstrate that large language models can generate cognitive tasks, specifically category learning tasks, that match the statistics of real-world tasks, thereby addressing the first challenge. We tackle the second challenge by deriving rational agents adapted to these tasks using the framework of meta-learning, leading to a class of models called ecologically rational meta-learned inference (ERMI). ERMI quantitatively explains human data better than seven other cognitive models in two different experiments. It additionally matches human behavior on a qualitative level: (1) it finds the same tasks difficult that humans find difficult, (2) it becomes more reliant on an exemplar-based strategy for assigning categories with learning, and (3) it generalizes to unseen stimuli in a human-like way. Furthermore, we show that ERMI's ecologically valid priors allow it to achieve state-of-the-art performance on the OpenML-CC18 classification benchmark., Comment: 22 pages (8 pages of main text, 4 pages of references, and 10 pages of appendix), 7 figures, and 4 Tables

Large language models (LLMs) are being increasingly incorporated into scientific workflows. However, we have yet to fully grasp the implications of this integration. How should the advent of large language models affect the practice of science? For this opinion piece, we have invited four diverse groups of scientists to reflect on this query, sharing their perspectives and engaging in debate. Schulz et al. make the argument that working with LLMs is not fundamentally different from working with human collaborators, while Bender et al. argue that LLMs are often misused and over-hyped, and that their limitations warrant a focus on more specialized, easily interpretable tools. Marelli et al. emphasize the importance of transparent attribution and responsible use of LLMs. Finally, Botvinick and Gershman advocate that humans should retain responsibility for determining the scientific roadmap. To facilitate the discussion, the four perspectives are complemented with a response from each group. By putting these different perspectives in conversation, we aim to bring attention to important considerations within the academic community regarding the adoption of LLMs and their impact on both current and future scientific practices.

As children grow older, they develop an intuitive understanding of the physical processes around them. Their physical understanding develops in stages, moving along developmental trajectories which have been mapped out extensively in previous empirical research. Here, we investigate how the learning trajectories of deep generative neural networks compare to children's developmental trajectories using physical understanding as a testbed. We outline an approach that allows us to examine two distinct hypotheses of human development - stochastic optimization and complexity increase. We find that while our models are able to accurately predict a number of physical processes, their learning trajectories under both hypotheses do not follow the developmental trajectories of children., Comment: Published as a conference paper at ICML 2023

Humans possess the ability to identify and generalize relevant features of natural objects, which aids them in various situations. To investigate this phenomenon and determine the most effective representations for predicting human behavior, we conducted two experiments involving category learning and reward learning. Our experiments used realistic images as stimuli, and participants were tasked with making accurate decisions based on novel stimuli for all trials, thereby necessitating generalization. In both tasks, the underlying rules were generated as simple linear functions using stimulus dimensions extracted from human similarity judgments. Notably, participants successfully identified the relevant stimulus features within a few trials, demonstrating effective generalization. We performed an extensive model comparison, evaluating the trial-by-trial predictive accuracy of diverse deep learning models' representations of human choices. Intriguingly, representations from models trained on both text and image data consistently outperformed models trained solely on images, even surpassing models using the features that generated the task itself. These findings suggest that language-aligned visual representations possess sufficient richness to describe human generalization in naturalistic settings and emphasize the role of language in shaping human cognition.

Large language models are powerful systems that excel at many tasks, ranging from translation to mathematical reasoning. Yet, at the same time, these models often show unhuman-like characteristics. In the present paper, we address this gap and ask whether large language models can be turned into cognitive models. We find that -- after finetuning them on data from psychological experiments -- these models offer accurate representations of human behavior, even outperforming traditional cognitive models in two decision-making domains. In addition, we show that their representations contain the information necessary to model behavior on the level of individual subjects. Finally, we demonstrate that finetuning on multiple tasks enables large language models to predict human behavior in a previously unseen task. Taken together, these results suggest that large, pre-trained models can be adapted to become generalist cognitive models, thereby opening up new research directions that could transform cognitive psychology and the behavioral sciences as a whole.

Everything else being equal, simpler models should be preferred over more complex ones. In reinforcement learning (RL), simplicity is typically quantified on an action-by-action basis -- but this timescale ignores temporal regularities, like repetitions, often present in sequential strategies. We therefore propose an RL algorithm that learns to solve tasks with sequences of actions that are compressible. We explore two possible sources of simple action sequences: Sequences that can be learned by autoregressive models, and sequences that are compressible with off-the-shelf data compression algorithms. Distilling these preferences into sequence priors, we derive a novel information-theoretic objective that incentivizes agents to learn policies that maximize rewards while conforming to these priors. We show that the resulting RL algorithm leads to faster learning, and attains higher returns than state-of-the-art model-free approaches in a series of continuous control tasks from the DeepMind Control Suite. These priors also produce a powerful information-regularized agent that is robust to noisy observations and can perform open-loop control.

Large language models have shown tremendous performance in a variety of tasks. In-context learning -- the ability to improve at a task after being provided with a number of demonstrations -- is seen as one of the main contributors to their success. In the present paper, we demonstrate that the in-context learning abilities of large language models can be recursively improved via in-context learning itself. We coin this phenomenon meta-in-context learning. Looking at two idealized domains, a one-dimensional regression task and a two-armed bandit task, we show that meta-in-context learning adaptively reshapes a large language model's priors over expected tasks. Furthermore, we find that meta-in-context learning modifies the in-context learning strategies of such models. Finally, we extend our approach to a benchmark of real-world regression problems where we observe competitive performance to traditional learning algorithms. Taken together, our work improves our understanding of in-context learning and paves the way toward adapting large language models to the environment they are applied purely through meta-in-context learning rather than traditional finetuning.

Large language models are transforming research on machine learning while galvanizing public debates. Understanding not only when these models work well and succeed but also why they fail and misbehave is of great societal relevance. We propose to turn the lens of computational psychiatry, a framework used to computationally describe and modify aberrant behavior, to the outputs produced by these models. We focus on the Generative Pre-Trained Transformer 3.5 and subject it to tasks commonly studied in psychiatry. Our results show that GPT-3.5 responds robustly to a common anxiety questionnaire, producing higher anxiety scores than human subjects. Moreover, GPT-3.5's responses can be predictably changed by using emotion-inducing prompts. Emotion-induction not only influences GPT-3.5's behavior in a cognitive task measuring exploratory decision-making but also influences its behavior in a previously-established task measuring biases such as racism and ableism. Crucially, GPT-3.5 shows a strong increase in biases when prompted with anxiety-inducing text. Thus, it is likely that how prompts are communicated to large language models has a strong influence on their behavior in applied settings. These results progress our understanding of prompt engineering and demonstrate the usefulness of methods taken from computational psychiatry for studying the capable algorithms to which we increasingly delegate authority and autonomy.

Meta-learning is a framework for learning learning algorithms through repeated interactions with an environment as opposed to designing them by hand. In recent years, this framework has established itself as a promising tool for building models of human cognition. Yet, a coherent research program around meta-learned models of cognition is still missing. The purpose of this article is to synthesize previous work in this field and establish such a research program. We rely on three key pillars to accomplish this goal. We first point out that meta-learning can be used to construct Bayes-optimal learning algorithms. This result not only implies that any behavioral phenomenon that can be explained by a Bayesian model can also be explained by a meta-learned model but also allows us to draw strong connections to the rational analysis of cognition. We then discuss several advantages of the meta-learning framework over traditional Bayesian methods. In particular, we argue that meta-learning can be applied to situations where Bayesian inference is impossible and that it enables us to make rational models of cognition more realistic, either by incorporating limited computational resources or neuroscientific knowledge. Finally, we reexamine prior studies from psychology and neuroscience that have applied meta-learning and put them into the context of these new insights. In summary, our work highlights that meta-learning considerably extends the scope of rational analysis and thereby of cognitive theories more generally.

As children grow older, they develop an intuitive understanding of the physical processes around them. They move along developmental trajectories, which have been mapped out extensively in previous empirical research. We investigate how children's developmental trajectories compare to the learning trajectories of artificial systems. Specifically, we examine the idea that cognitive development results from some form of stochastic optimization procedure. For this purpose, we train a modern generative neural network model using stochastic gradient descent. We then use methods from the developmental psychology literature to probe the physical understanding of this model at different degrees of optimization. We find that the model's learning trajectory captures the developmental trajectories of children, thereby providing support to the idea of development as stochastic optimization., Comment: Submitted to SVRHM at NeurIPS 2022

We study GPT-3, a recent large language model, using tools from cognitive psychology. More specifically, we assess GPT-3's decision-making, information search, deliberation, and causal reasoning abilities on a battery of canonical experiments from the literature. We find that much of GPT-3's behavior is impressive: it solves vignette-based tasks similarly or better than human subjects, is able to make decent decisions from descriptions, outperforms humans in a multi-armed bandit task, and shows signatures of model-based reinforcement learning. Yet we also find that small perturbations to vignette-based tasks can lead GPT-3 vastly astray, that it shows no signatures of directed exploration, and that it fails miserably in a causal reasoning task. These results enrich our understanding of current large language models and pave the way for future investigations using tools from cognitive psychology to study increasingly capable and opaque artificial agents.

The autoionization dynamics of superexcited superfluid He nanodroplets doped with Na atoms is studied by extreme-ultraviolet (XUV) time-resolved electron spectroscopy. Following excitation into the higher-lying droplet absorption band, the droplet relaxes into the lowest metastable atomic $1s2s$ $^{1,\,3}$S states from which Interatomic Coulombic Decay (ICD) takes places either between two excited He atoms or between an excited He atom and a Na atom attached to the droplet surface. Four main ICD channels are identified and their time constants are determined by varying the delay between the XUV pulse and a UV pulse that ionizes the initial excited state and thereby quenches ICD. The time constants for the different channels all fall in the range $\sim$1~ps indicating that the ICD dynamics are mainly determined by the droplet environment. A periodic modulation of the transient ICD signals is tentatively attributed to the oscillation of the bubble forming around the localized He excitation. The ICD efficiency depends on the total number of excited states in a droplet rather than the density of excited states pointing to a collective enhancement of ICD.

Equipping artificial agents with useful exploration mechanisms remains a challenge to this day. Humans, on the other hand, seem to manage the trade-off between exploration and exploitation effortlessly. In the present article, we put forward the hypothesis that they accomplish this by making optimal use of limited computational resources. We study this hypothesis by meta-learning reinforcement learning algorithms that sacrifice performance for a shorter description length (defined as the number of bits required to implement the given algorithm). The emerging class of models captures human exploration behavior better than previously considered approaches, such as Boltzmann exploration, upper confidence bound algorithms, and Thompson sampling. We additionally demonstrate that changing the description length in our class of models produces the intended effects: reducing description length captures the behavior of brain-lesioned patients while increasing it mirrors cognitive development during adolescence., Comment: NeurIPS 2022 final version

Interferometric pump-probe experiments in the extreme ultraviolet (XUV) domain are experimentally very challenging due to the high phase stability required between the XUV pulses. Recently, an efficient phase stabilization scheme was introduced for seeded XUV free electron lasers (FELs) combining shot-to-shot phase modulation with lock-in detection. This method stabilized the seed laser beampath on the fundamental ultraviolet wavelength to a high degree. Here, we extend this scheme including the stabilization of the XUV beampath, incorporating phase fluctuations from the FEL high gain harmonic generation process. Our analysis reveals a clear signal improvement with the new method compared to the previous stabilization scheme.

The relaxation dynamics of superexcited superfluid He nanodroplets is thoroughly investigated by means of extreme-ultraviolet (XUV) femtosecond electron and ion spectroscopy complemented by time-dependent density functional theory (TDDFT). Three main paths leading to the emission of electrons and ions are identified: Droplet autoionization, pump-probe photoionization, and autoionization induced by re-excitation of droplets relaxing into levels below the droplet ionization threshold. The most abundant product of both droplet autoionization and photoionization is He$_2^+$, whereas the delayed appearance of He$^+$ is indicative of the ejection of excited He atoms from the droplets. The state-resolved time-dependent photoelectron spectra reveal that intermediate excited states of the droplets are populated in the course of the relaxation, terminating in the lowest-lying metastable singlet and triplet He atomic states. The slightly faster relaxation of the triplet state compared to the singlet state is in agreement with the simulation showing faster formation of a bubble around a He atom in the triplet state.

Collinear double-pulse seeding of the High-Gain Harmonic Generation (HGHG) process in a free-electron laser (FEL) is a promising approach to facilitate various coherent nonlinear spectroscopy schemes in the extreme ultraviolet (XUV) spectral range. However, in collinear arrangements using a single nonlinear medium, temporally overlapping seed pulses may introduce nonlinear mixing signals that compromise the experiment at short time delays. Here, we investigate these effects in detail by extending the analysis described in a recent publication (Wituschek et al., Nat. Commun., 11, 883, 2020). High-order fringe-resolved autocorrelation and wave-packet interferometry experiments at photon energies > $23\,$eV are performed, accompanied by numerical simulations. It turns out that both the autocorrelation and the wave-packet interferometry data are very sensitive to saturation effects and can thus be used to characterize saturation in the HGHG process. Our results further imply that time-resolved spectroscopy experiments are feasible even for time delays smaller than the seed pulse duration., Comment: This is accepted version of the article. The Version of Record is available online at https://doi.org/10.1364/OE.401249

The effects of high pulse intensity and chirp on two-dimensional electronic spectroscopy signals are experimentally investigated in the highly non-perturbative regime using atomic rubidium vapor as clean model system. Data analysis is performed based on higher-order Feynman diagrams and non-perturbative numerical simulations of the system response. It is shown that higher-order contributions may lead to a fundamental change of the static appearance and beating-maps of the 2D spectra and that chirped pulses enhance or suppress distinct higher-order pathways. We further give an estimate of the threshold intensity beyond which the high-intensity effects become visible for the system under consideration., Comment: 35 pages, 15 figures

The recent development of ultrafast extreme ultraviolet (XUV) coherent light sources bears great potential for a better understanding of the structure and dynamics of matter. Promising routes are advanced coherent control and nonlinear spectroscopy schemes in the XUV energy range, yielding unprecedented spatial and temporal resolution. However, their implementation has been hampered by the experimental challenge of generating XUV pulse sequences with precisely controlled timing and phase properties. In particular, direct control and manipulation of the phase of individual pulses within a XUV pulse sequence opens exciting possibilities for coherent control and multidimensional spectroscopy, but has not been accomplished. Here, we overcome these constraints in a highly time-stabilized and phase-modulated XUV-pump, XUV-probe experiment, which directly probes the evolution and dephasing of an inner subshell electronic coherence. This approach, avoiding any XUV optics for direct pulse manipulation, opens up extensive applications of advanced nonlinear optics and spectroscopy at XUV wavelengths., Comment: This is the accepted version of the article. The Version of Record is available online at 10.1038/s41467-020-14721-2

Recent work applying multidimentional coherent electronic spectroscopy at dilute samples in the gas phase is reviewed. The development of refined phase-cycling approaches with improved sensitivity has opened-up new opportunities to probe even dilute gas-phase samples. In this context, first results of 2-dimensional spectroscopy performed at doped helium droplets reveal the femtosecond dynamics upon electronic excitation of cold, weakly-bound molecules, and even the induced dynamics from the interaction with the helium environment. Such experiments, offering well-defined conditions at low temperatures, are potentially enabling the isolation of fundamental processes in the excitation and charge transfer dynamics of molecular structures which so far have been masked in complex bulk environments., Comment: Invited Review Article

We compare the acquisition of knowledge in humans and machines. Research from the field of developmental psychology indicates, that human-employed hypothesis are initially guided by simple rules, before evolving into more complex theories. This observation is shared across many tasks and domains. We investigate whether stages of development in artificial learning systems are based on the same characteristics. We operationalize developmental stages as the size of the data-set, on which the artificial system is trained. For our analysis we look at the developmental progress of Bayesian Neural Networks on three different data-sets, including occlusion, support and quantity comparison tasks. We compare the results with prior research from developmental psychology and find agreement between the family of optimized models and pattern of development observed in infants and children on all three tasks, indicating common principles for the acquisition of knowledge.

Human decision-making deviates from the optimal solution, that maximizes cumulative rewards, in many situations. Here we approach this discrepancy from the perspective of bounded rationality and our goal is to provide a justification for such seemingly sub-optimal strategies. More specifically we investigate the hypothesis, that humans do not know optimal decision-making algorithms in advance, but instead employ a learned, resource-bounded approximation. The idea is formalized through combining a recently proposed meta-learning model based on Recurrent Neural Networks with a resource-bounded objective. The resulting approach is closely connected to variational inference and the Minimum Description Length principle. Empirical evidence is obtained from a two-armed bandit task. Here we observe patterns in our family of models that resemble differences between individual human participants., Comment: Final version for CogSci 2019

Two-dimensional electronic spectroscopy (2DES) is one of the most powerful spectroscopic techniques, capable of attaining a nearly complete picture of a quantum system including its couplings, quantum coherence properties and its real-time dynamics. While successfully applied to a variety of condensed phase samples, high precision experiments on isolated quantum systems in the gas phase have been so far precluded by insufficient sensitivity. However, such experiments are essential for a precise understanding of fundamental mechanisms and to avoid misinterpretations, e.g. as for the nature of quantum coherences in energy trans-port. Here, we solve this issue by extending 2DES to isolated nanosystems in the gas phase prepared by helium nanodroplet isolation in a molecular beam-type experiment. This approach uniquely provides high flexibility in synthesizing tailored, quantum state-selected model systems of single and many-body properties. For demonstration, we deduce a precise and conclusive picture of the ultrafast coherent dynamics in isolated high-spin Rb2 molecules and present for the first time a dynamics study of the system-bath interaction between a single molecule (here Rb3) and a superfluid helium environment. The results demonstrate the unique capacity to elucidate prototypical interactions and dynamics in tailored quantum systems and bridges the gap to experiments in ultracold quantum science.

We introduce the concept of phase-synchronous undersampling in nonlinear spectroscopy. The respective theory is presented and validated experimentally in a phase-modulated quantum beat experiment by sampling high phase modulation frequencies with low laser repetition rates. The advantage of undersampling in terms of signal quality and reduced acquisition time is demonstrated and breakdown conditions are identified. The presented method is particularly beneficial for experimental setups with limited signal/detection rates., Comment: 4 pages, 5 figures

A novel technique to sensitively and selectively isolate multiple-quantum coherences in a femtosecond pump-probe setup is presented. Detecting incoherent observables and imparting lock-in amplification, even weak signals of highly dilute samples can be acquired. Applying this method, efficient isolation of one- and two-photon transitions in a rubidium-doped helium droplet beam experiment is demonstrated and collective resonances up to fourth order are observed in a potassium vapor for the first time. Our approach provides new perspectives for coherent experiments in the deep UV and novel multidimensional spectroscopy schemes, in particular when selective detection of particles in dilute gas-phase targets is possible.